Home

Lab 6 - Logistic Regression

Parthasarathy Rajendiran
8883828

---

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import plotly
import plotly.graph_objects as go
plotly.offline.init_notebook_mode()

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score, recall_score, f1_score, accuracy_score, confusion_matrix, classification_report
from sklearn.metrics import precision_recall_curve

Loading iris dataset

iris_X, iris_y = load_iris(return_X_y=True, as_frame=True)

display(iris_X)
display(iris_y.value_counts())

	sepal length (cm)	sepal width (cm)	petal length (cm)	petal width (cm)
0	5.1	3.5	1.4	0.2
1	4.9	3.0	1.4	0.2
2	4.7	3.2	1.3	0.2
3	4.6	3.1	1.5	0.2
4	5.0	3.6	1.4	0.2
...	...	...	...	...
145	6.7	3.0	5.2	2.3
146	6.3	2.5	5.0	1.9
147	6.5	3.0	5.2	2.0
148	6.2	3.4	5.4	2.3
149	5.9	3.0	5.1	1.8

150 rows × 4 columns

target
0    50
1    50
2    50
Name: count, dtype: int64

Setting Target label

tgt_names = load_iris().target_names
display(tgt_names)

array(['setosa', 'versicolor', 'virginica'], dtype='<U10')

# The target should be virginica or non-virginica
target_label = tgt_names[iris_y] == 'virginica'

Train Test Split

X_train, X_test, y_train, y_test = train_test_split(iris_X, target_label, random_state=8883828)

print(f'The dataset of size {iris_X.shape} has been splitted into')
print(f'Training set  : {X_train.shape}')
print(f'Testing set   : {X_test.shape}')

The dataset of size (150, 4) has been splitted into
Training set  : (112, 4)
Testing set   : (38, 4)

Model Building

log_reg = LogisticRegression(random_state=8883828)
log_reg.fit(X_train, y_train)

LogisticRegression(random_state=8883828)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Model Evaluation

y_pred = log_reg.predict(X_test)
y_pred_proba = log_reg.predict_proba(X_test)

def show_metrics(y_actual, y_predicted):
    # Confusion Matrix
    conf_mat = confusion_matrix(y_actual, y_predicted)
    tn, fp, fn, tp = conf_mat.flatten()
    print(f'True Positive : {tp:>4}\t\tFalse Positive : {fp:>4}\nTrue Negative : {tn:>4}\t\tFalse Negative : {fn:>4}')
    
    # Metrics
    precision = precision_score(y_actual, y_predicted)
    recall = recall_score(y_actual, y_predicted)
    f1 = f1_score(y_actual, y_predicted)
    acc = accuracy_score(y_actual, y_predicted)

    print(
        f'\nPrecision : {precision:.4}\t\tF1 score  : {f1:.4}\nRecall    : {recall:.4}\t\t\tAccuracy  : {acc:.4%}'
    )

    # Plotting
    group_names = ['True -ve', 'False +ve', 'False -ve', 'True +ve']
    group_counts = [tn, fp, fn, tp]
    group_percentages = ['{0:.2%}'.format(i) for i in conf_mat.flatten()/np.sum(conf_mat)]
    labels = [
        f'{n}\n\n{c}\n\n{p}' for n, c, p in zip(group_names, group_counts, group_percentages)
    ]
    labels = np.asarray(labels).reshape(2, 2)

    plt.figure(figsize=(4.5, 3))
    ax = sns.heatmap(conf_mat, annot=labels, fmt='', cmap='Greens', linewidths=0.5, linecolor='Black')
    for _, spine in ax.spines.items():
        spine.set_visible(True)
    ax.set_title('Confusion Matrix')
    ax.set_xlabel('Predicted Values')
    ax.set_ylabel('Actual Values')
    ax.xaxis.set_ticklabels(['False', 'True'])
    ax.yaxis.set_ticklabels(['False', 'True'])
    plt.show()
    print(classification_report(y_actual, y_predicted))

show_metrics(y_test, y_pred)

True Positive :   13		False Positive :    1
True Negative :   24		False Negative :    0

Precision : 0.9286		F1 score  : 0.963
Recall    : 1.0			Accuracy  : 97.3684%

              precision    recall  f1-score   support

       False       1.00      0.96      0.98        25
        True       0.93      1.00      0.96        13

    accuracy                           0.97        38
   macro avg       0.96      0.98      0.97        38
weighted avg       0.98      0.97      0.97        38

i. Failure modes: in which data instances is the model wrong?

From the confusion matrix, we can observe there is a False Positive which means one record was classified as Virginica where it is actually Non-virginica

# Combining the columns for analysis
analysis_df = X_test
analysis_df['target'] = iris_y.apply(lambda x: tgt_names[x])
analysis_df['actual_y'] = y_test
analysis_df['predicted_y'] = y_pred
analysis_df['false_pred_proba'] = y_pred_proba[:, 0]
analysis_df['true_pred_proba'] = y_pred_proba[:, 1]

# The False Positive record
analysis_df[analysis_df['actual_y'] != analysis_df['predicted_y']]

	sepal length (cm)	sepal width (cm)	petal length (cm)	petal width (cm)	target	actual_y	predicted_y	false_pred_proba	true_pred_proba
77	6.7	3.0	5.0	1.7	versicolor	False	True	0.381424	0.618576

As per the the trained logistic regression model, this record has 38% probability to be False and 62% probability to be True.
However, in reality this record is False (Non-Virginica). So, in this instance the model is wrong.

ii. Are there any shared properties for these cases? (1 point)

As there is only one misclassification (false positive) and there is no false negative, no shared properties can be found.

precision, recall, thresholds = precision_recall_curve(analysis_df['actual_y'], analysis_df['true_pred_proba'])

# Plotting Precision Recall curve
fig = go.Figure(data=[
    go.Scatter(x=recall, y=precision, mode='lines', name='Precision-Recall Curve')
])

fig.update_layout(
    title='Precision-Recall Curve',
    xaxis_title='Recall',
    yaxis_title='Precision',
    yaxis_range=[0, 1],
    height=500, width=500
)

fig.show()

With the above Precision-Recall curve, we can observe that we cannot have both Precision and Recall as 1.

iii. How is the model doing across a set of evaluation metrics: accuracy and confusion metric. (1 point)

Accuracy
The model performs well with the Accuracy of 97.37%
i.e., The model is correctly predicting 97 times for 100 trials.

Confusion Matrix
True Positives: The model correctly predicted 13 instances as Virginica.
True Negatives: The model correctly predicted 24 instances as Non-Virginica.
False Positives: The model incorrectly predicted 1 instance as Virginica where it is actually Non-Virginica.
False Negative: The model doesn't predict any Non-Virginica as Virginica

---